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VALIDITY  ANALYSIS  OF  THE  GROUP  AWARENESS  TEST  OF  THE 
DIFFERENTIAL  OFFICER  BATTERY 


j  BACKGROUND 

"<^The  Group  Awareness  Test  (GAT)  was  developed  as  part  of  an  extensive 
program  of  research  to  determine  the  extent  to  which  ability  to  meet  the 
psychological  demands  of  combat,  administrative,  and  technical  officer 
assignments  csn  be  differentially  predicted  by  psychological  measures./ 

As  the  first  step  of  this  program,  an  extensive  battery  of  experimental 
tests  was  constructed  and  administered  to  over  6000  offlcera.at  entry 
on  active  duty  in  1958.  On  the  basis  of  item  analysis  against  perfor¬ 
mance  ratings  after  about  18  months  of  service,  the  earlier  battery — the 
Differential  Officer  Leadership  Battery  (DOL) — was  revised  and  shortened 
to  form  the  Differential  Officer  Battery  (DOB).  In  1961  and  1962,  the 
DOB  was  administered  to  about  4000  officers  at  entry  on  active  duty. 

'internal  analysis  of  the  Group  Awareness  Test  was  completed  and  re¬ 
ported  in  Research  Memorandum  68-10.— 7  Subsequently,  the  scores  derived 
tram  the  Internal  analysis  were  Included  in  a  factor  analysis  of  149 
subscores  from  the  Differential  Officer  Battery.  In  this  analysis  sev¬ 
eral  factors  included  only  GAT  scores.  Because  of  this  finding — and 
the  unique  nature  of  items  of  the  GAT — further  specific  validation  studies 
were  undertaken. \The  present  publication  covers  work  completed  on  the 
further  validation?  — — 


OBJECTIVE 

The  Group  Awareness  Test  was  Included  in  the  experimental  DOL  in  an 
effort  to  measure  the  officer's  empathy  or  ability  to  estimate  the  opinions 
of  peers  and  subordinates.  As  first  constructed ^  ,  the  GAT  consisted  of 
two  forms,  A  (DA  PT  3400)  and  B  (DA  PT  3401),  each  composed  of  75  job- 
related  attltudlnal  statements.  When  the  DOL  was  revised  and  shortened 
to  form  the  DOB,  the  GAT  was  reduced  to  a  single  form  of  75  items 


^  Smith,  K.  H.  Internal  analysis  of  the  Group  Awareness  Test  for  the 

Differential  Officer  Battery.  ARI  Research  Memorandtan  68-10.  September 
1968. 

a 

Bornsteln,  H. ,  R.  Sadacca,  and  R.  Phillips.  Development  of  differential 
officer  leadership  experimental  test  battery.  ARI  Research  Memorandum 
57-27.  December  1957. 


(DA  PT  4093) i'  .  The  Items  are  all  statements  of  opinion  or  attitude 
with  regard  to  work  habit a,  leader-follower  relationships,  human  nature, 
and  various  kinds  of  pursuits  and  activities.  The  examinee  is  Instructed 
to  estimate  for  each  statenent  what  proportions  of  representative  groups 
of  NCOs,  recruits,  and  ROTC  graduates  would  endorse  the  statenent.  There 
are  thus  three  responses  to  each  of  the  75  Itens,  producing  a  total  of  225 
scorable  responses.  The  format  for  responding  is  such  that  each  item  on 
the  answer  sheet  Is  followed  by  three  rows  of  five  alternatives.  In  each 
row,  the  alternatives  are  represented  by  the  letters  A,  B,  C,  D,  E.  At 
the  top  of  each  page  of  the  test  booklet,  the  following  code  for  the 
alternatives  is  printed:  A  -  Very  few  (02  -  192);  B  •  Few  (202  -  392); 

C  -  About  half  (402  -  592);  D  -  Many  (602  -  792);  E  -  Very  many  (802  - 

1002). 

In  the  Internal  analysis,  it  was  difficult  to  find  consistent  and 
stable  clusters  beyond  a  general  Social  Desirability  cluster  and  a 
Cynicism  cluster,  both  of  which  seem  to  be  general  across  the  three  esti¬ 
mation  populations.  Further,  it  was  difficult  to  determine  whether  a 
score  based  on  the  accuracy  of  the  examinee’s  estimates  of  the  groups' 
real  responses  or  a  score  based  on  simply  his  attitude  about  what  the  re¬ 
sponses  would  be  was  the  most  significant  indicator  of  leadership  behavior 
potential.  As  a  result,  the  24  scores  which  were  finally  derived  from  the 
GAT  were  computed  as  simple  summation  scores  and  also  as  deviation  accuracy 
scores. 

In  the  present  analysis,  these  48  scores  were  utilized  along  with 
some  others  specially  computed  for  the  purpose.  The  specially  computed 
scores  were  of  two  general  kinds.  The  first  kind  Involved  differences 
In  the  examinee' s  estimates  of  the  responses  which  would  be  made  by  the 
three  estimation  populations.  There  were  36  such  "difference"  scores. 

The  second  kind  of  score  specially  computed  for  this  analysis  Involved 
clustering  of  item  types  on  a  different  basis  than  those  derived  from 
the  internal  analysis.  Twelve  such  scores  were  obtained. 

The  present  research  memorandum  presents  the  analysis  of  the  96  scores 
from  the  GAT  described  above  for  their  validity  in  predicting  performance 
at  the  Officer  Evaluation  Center  (OEC).k 

T 

PROCEDURES 


SAMPLE 

For  this  study  the  sample  Included  some  735  cases  for  whom  complete 
DOB  records  were  available  and  for  whom  complete  records  of  performance 


^  HUlamin,  L.  P.  Prediction  of  officer  performance.  ARI  Technical 
Research  Report  1134.  March  1964. 


at  the  OSC  were  also  available.  This  same  saaple  serves  as  the  validity 
analysis  sample  in  all  Instances  where  predictors  are  being  validated 
against  performance  at  OSC. 


GAT  SCALE  SCORES 

Of  the  scores  analyzed  in  the  present  study,  48  came  directly  from 
the  internal  analysis  reported  in  Research  Memorandum  68-lOi-'  .  Four 
additional  scales  were  constructed  utilizing  new  item  clusterings.  A 
general  Social  Desirability  score  was  computed  Including  all  items  on 
which  the  three  estimation  populations  had  actually  responded  prepon¬ 
derantly  on  either  end  of  the  scale.  Three  other  item  cluster  scales 
were  made  up  Including  those  items  on  which  members  of  a  specific  esti¬ 
mation  population  had  responded  in  a  way  substantially  different  from 
the  other  two  estimation  populations.  These  scales  are  titled  NCO  Unique, 
Recruit  Unique,  and  ROTC  Unique.  The  item  clusters  making  up  these  scales 
are  listed  in  Table  A-l. 

Bach  of  the  simple  item  sum  scale  scores  can  be  conceived  as  an 
estimate  by  the  examinee  of  the  typical  attitude  of  members  of  the  esti¬ 
mation  population  with  regard  to  the  subject  matter  represented  by  the 
item  cluster.  For  example,  the  Good  Work  Habits  scale  score  for  the  NCO 
estimation  population  can  be  considered  as  the  examinee's  estimate  of 
what  the  average  NCO's  attitude  is  toward  the  value  of  hard  work.  When 
this  score  is  compared  with  the  same  scale  score  for  Recruits  end  for 
recent  ROTC  graduates,  we  can  begin  to  get  some  feeling  for  the  examinee's 
relative  estimates  of  the  attitudes  of  these  three  estimation  populations. 
In  order  to  determine  the  predictive  validity  of  these  differences  in 
attitude  estimation,  each  of  the  item  sum  scale  scores  was  utilized  in 
computing  difference  scores.  Twelve  item  sum  scores  were  so  utilized — 
the  eight  derived  from  the  original  internal  analysis  and  the  four  com¬ 
puted  specifically  for  the  present  validity  study.  For  each  of  these  12 
item  clusters,  three  difference  scores  were  computed:  1)  by  subtracting 
the  Recruit  estimation  from  the  NCO  estimation  total,  2)  by  subtracting 
the  ROTC  estimation  total  from  the  NCO  total,  and  3)  by  subtracting  the 
ROTC  estimation  total  from  the  Recruit  total  .- 


OSC  PERFORMANCE  SCORES 

At  the  OBC,  IS  tasks  were  performed  in  a  simulated  combat  setting 
over  a  three-day  period-^.  Each  task  yielded  a  total  score  along  with 
several  other  scores  more  specific  to  performance  of  part  of  the  task  or 


^  Smith,  K.  H.  Internal  analysis  of  the  Group  Awsreness  Test  for  the 
Differential  Officer  Battery.  ARI  Research  Memorandum  68-10.  September 
1968. 
e 

Helms,  V.  H.,  L.  P.  Hlllmsin,  and  Frances  C.  Grafton.  Dimensions  of 
leadership  in  a  simulated  combat  situation.  ARI  Technical  Research  Re- 
pert  1172.  July  1971. 
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to  aspects  of  the  examinee's  task  behavior.  There  were  342  Individual 
performance  scores  In  total.  These  were  factor  analyzed  and  yielded 
thirty  factors  of  performance  more  or  less  specific  to  each  of  the  15 
tasks.  In  a  separate  analysis,  eight  factors  which  represented  behavior 
over  several  of  the  tasks  were  derived.  Using  correlation  of  sums  with 
unit  weights  in  most  instances  and  test  selection  procedures  in  others, 
combinations  of  scores  were  computed  to  provide  a  score  for  each  exam¬ 
inee  on  each  of  the  38  factors. 

These  38  factor  scores,  along  with  the  15  task  total  scores,  served 
as  the  criterion  scores  for  validation  of  the  GAT  as  a  predictor  of  per¬ 
formance.  It  should  be  noted  that  none  of  these  three  sets  of  OEC  scores 
is  totally  independent  of  the  others. 


RESULTS 

OEC  CROSS-TASK  FACTOR  SCORE  PREDICTION 

In  general,  the  results  of  the  product  moment  correlational  analysis 
between  the  y6  GAT  scores  and  the  8  OEC  cross-task  factor  scores  are  dis¬ 
appointing.  The  highest  correlation  which  resulted  was  .17.  With  a 
sample  of  735  cases,  a  correlation  of  approximately  .08  reaches  statistical 
significance  at  the  .05  level.  However,  in  terms  of  the  proportion  of 
variance  accounted  for,  the  level  necessary  for  practical  significance 
would  be  considerably  higher. 

In  order  to  analyze  the  pattern  of  correlations,  only  correlation 
coefficients  of  .10  or  higher  in  absolute  magnitude  were  considered.  Only 
two  of  the  cross-task  factors  showed  any  consistent  relationship  with  the 
GAT  scores.  These  were:  factor  1,  Technical-managerial  leadership;  and 
factor  7,  Tactical  skills.  None  of  the  other  cross-task  factors  showed 
more  than  4  coefficients  of  .10  or  above,  and  none  of  these  was  above  .12. 
This  suggests  strongly  that  chance  was  involved  in  these  correlations. 
Tables  1  and  2  show  the  pattern  of  correlations  for  factors  1  and  7  with 
the  96  GAT  scores.  In  the  tables,  coefficients  of  less  than  .10  in  abso¬ 
lute  magnitude  have  been  deleted  for  purposes  of  clarity. 

From  a  review  of  Tables  1  and  2,  we  see  that  factor  7,  Tactical  skills, 
is  most  consistently  predicted.  Nearly  all  the  original  sum  and  accuracy 
scores  are  related,  as  are  quite  a  few  of  the  scale  difference  scores.  It 
is  interesting  that  the  sum  scores  are  positively  related  in  all  but  one 
case.  On  the  other  hand,  the  accuracy  scores  are  negatively  related  for 
recruit  estimates,  positively  related,  generally,  for  R0TC  estimates,  and 
split  in  direction  for  NC0  estimates.  Twelve  of  fourteen  correlations 
for  scale  difference  scores  are  negative.  This  suggests  that  large  dif¬ 
ferences  in  the  perception  of  the  estimation  groups  that  favor  NCO's  over 
recruits  or  R0TC  graduates,  or  that  favor  recruits  over  R0TC  graduates, 
are  associated  with  poor  performance  in  activities  involved  in  the  tacti¬ 
cal  skills  factor  score. 
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Factor  1,  Technical-managerial  leadership,  is  ouch  loss  consistently 
predicted.  Moot  of  tbo  coefficients  of  .10  or  higher  reflect  correlation 
with  the  original  accuracy  acoreo,  and  to  a  leaser  extent  with  the  original 
mm  acoraa.  It  la  of  particular  lntereat  that  in  ovary  caaa  where  both 
factors  1  and  7  correlate  with  a  GAT  score  at  the  level  of  .10  or  higher 
the  relationships  are  in  opposite  directions.  Since  this  occurs  16  tines, 
it  seems  safe  to  conclude  that  the  characteristics  measured  by  the  GAT  re¬ 
late  to  performance  in  the  technical-managerial  area  and  the  tactical 
skills  area  in  opposite  ways. 


OK  TASK-SPECIFIC  FACTOR  SCORE  PREDICTION 

The  results  of  correlating  the  30  task-specific  OK  performance 
factors  with  the  96  GAT  scores  were  similar  to  those  for  the  cross-task 
factors.  Four  task-specific  factors  showed  relatively  consistent  patterns 
of  correlation.  These  were:  factor  2,  Mission  Effectiveness  in  the 
Coaenmlcation  Exhibit  task;  factor  8,  Mission  Effectiveness  in  the  Auto- 
motive  Inspection  task;  factor  14,  Mission  Effectiveness  in  the  Highway 
Traffic  Plan  task;  and  factor  18,  Combat  Persistence  versus  Technical 
Persistence  (one  of  two  factors  in  the  30  factor  solution  with  loadings 
across  more  than  one  task). 

While  none  of  the  coefficients  in  Tables  3  and  4  is  very  high  (a 
maximum  of  .16),  they  show  consistency  with  those  found  in  Tables  1  and 
2.  Among  the  original  48  sum  and  accuracy  scores  there  is  sufficient 
overlap  to  compare  the  results  of  Table  1  and  TaMe  3.  Of  20  Instances 
where  coefficients  of  .10  or  higher  occur  for  both  cross-task  factor  7 
and  factor  18  of  the  30  factor  solution,  18  show  the  same  sign.  This 
strongly  suggests  that  the  pattern  of  relationships  of  GAT  scores  with 
combat  or  tactical  performance  is  the  same. 

The  pattern  of  relationships  shown  in  Table  3  for  factors  2,  8,  and 
14  is  remarkably  consistent.  In  only  two  instances  out  of  36  cases,  where 
at  least  two  of  these  factors  show  coefficients  of  .10  or  higher  with  a 
single  GAT  score,  is  there  a  difference  in  signs.  This  pattern  of  signs 
agrees  almost  completely  with  that  shown  for  factor  1  in  Table  1.  This 
would  be  expected  since  factors  2,  8,  and  14  from  the  thirty  task-specific 
factors  all  deal  with  performance  in  technical  areas  end  factor  1  of  the 
8  cross-task  factors  is  called  technical-managerial  leadership. 

While  factors  2,  8,  and  14  show  consistent  patterns  in  Table  4, 
factor  1  does  not  show  enough  coefficients  in  Table  2  for  comparison. 

The  reverse  is  true  for  the  comparison  of  factor  7  in  Table  2  as  compared 
with  factor  18  in  Table  4.  In  general,  it  seams  safe  to  say  that  the 
patterns  shown  in  Tables  3  and  4  lead  to  a  conclusion  quite  similar  to 
that  drawn  from  Tables  1  and  2.  Tbs  characteristics  measured  by  the  GAT 
relate  to  performance  in  technical  areas  and  performance  in  combat  areas 
in  opposite  ways. 


OEC  TASK.  TOTAL  SCORE  PREDICTION 


The  correlations  between  OEC  teak  total  scores  and  the  96  GAT  variables 
rsveel  a  continuation  of  the  pattern  shown  by  the  other  analyses.  Here 
three  total  scores  show  consistent  relationships  with  GAT  scores.  In 
Tables  5  and  6»  the  total  scores'  for  Airfield  Layout  (AL)  and  Automotive 
Inspection  (AI)  show  patterns  of  correlations  with  the  sane  sign  as  those 
for  the  mission  effectiveness  factor  scores  from  these  two  tasks  (see 
factor  8  in  Table  3).  The  correlations  are  also  of  about  the  same  magni¬ 
tude.  The  total  score  for  the  Improper  Supply  Records  (ISR)  task  shows 
the  same  pattern  of  signs,  but  the  absolute  magnitude  of  the  relationships 
is  a  little  higher.  Many  of  these  relationships  are  in  the  low  to  middle 
20' s. 


The  general  pattern  Is  again  one  of  prediction  of  technical  perfor¬ 
mance.  One  might  wonder,  however,  why  the  ISR  task  total  score  should 
show  the  strongest  pattern  of  correlations  when  the  two  factors,  from 
the  thirty-factor  solution,  based  on  ISR  scores  did  not  show  consistent 
patterns.  The  answer  seems  to  lie  in  the  fact  that  the  ISR  task  required 
two  very  different  kinds  of  skills— one  technical  and  one  Interpersonal. 

The  majority  of  the  examinee's  time  in  this  task  was  spent  in  correcting 
erroneous  supply  records.  He  then  had  to  brief  an  angry  and  difficult 
officer  on  the  reasons  that  the  records  were  In  error.  The  two  task- 
specific  factors  both  Involved  only  acores  related  to  this  briefing. 

One  of  these,  factor  12,  was  based  primarily  on  his  poise  and  his  under¬ 
standing  of  the  principles  he  was  trying  to  explain.  The  other,  factor 
19,  was  based  mainly  on  his  tact  end  flexibility  in  handling  the  inter- 
peraonal  problems  presented  by  the  angry  officer.  To  the  extent  that 
comparison  was  possible  with  the  few  coefficients  above  .10  for  each  of 
these  factors.  It  appeared  that  the  former  was  compatible  with  the  gen¬ 
eral  pattern  of  GAT  correlations  predicting  technical  performance  while 
the  latter  was  not.  The  total  task  score  was  more  completely  reflective 
of  performance  in  the  technical  area  of  understanding  the  record  keeping 
principles  and  the  use  of  them  in  correcting  the  supply  records,  brief¬ 
ing  the  officer,  and  writing  a  clear  report.  The  total  score  was  apparent¬ 
ly  not  as  much  affected  by  the  more  specifically  interpersonal  ratings  of 
tact  and  flexibility. 

Table  7  contrasts  the  pattern  of  correlations  with  GAT  scores  obtained 
for  the  Communications  Exhibit  (CE)  total  score  with  those  obtained  for 
the  March  Order  (MO)  total  score.  Again  CE  shows  the  technical  performance 
pattern.  To  the  extent  that  comparison  is  possible  the  MO  pattern  Is  com¬ 
patible  with  the  tactical  or  combat  pattern  . 

Since  none  of  tbs  individual  correlation  coefficients  was  very  large, 
it  was  felt  that,  perhaps,  some  weighted  combination  of  scores  might  pre¬ 
dict  OEC  performance  at  a  more  respectable  level.  As  a  result,  a  test 
selection  procedure  was  applied  using  the  ISR  total  score  as  the  dependent 
variable  end  the  48  original  sum  and  accuracy  scores  ae  predictors.  An 
addition  of  at  least  one  percent  of  variance  accounted  for  in  the  multiple 
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correlation  wee  utilised  as  the  minimal  criterion  for  inclusion  of 
additional  predictors.  Once  the  predictor  with  the  highest  aero-order 
correlation  had  been  selected,  none  of  the  other  predictors  met  the 
minimal  criterion  for  inclusion  in  a  multiple  correlation.  This  pro¬ 
cedure  was  repeated  using  the  AL  total  score  as  dependent  variable, 
with  the  same  result.  Apparently  the  48  sum  and  accuracy  scores 
overlap  almost  completely  in  the  variance  they  account  for  in  the 
performance  scores. 


CONCLUSIONS 

There  is  sufficient  regularity  to  the  patterns  of  relationship  shown 
by  the  GAT  scores  to  performance  in  the  OEC  situational  tasks  to  suggest 
that  further  research  could  be  revealing.  If  certain  assumptions  were 
made  it  might  be  possible  to  tie  these  results  to  some  basic  conclusions 
about  leadership  that  seem  fairly  generally  accepted  among  industrial 
psychologists.  If  we  should  assume  that  the  GAT  is  measuring  consistent 
dispositional  characteristics  of  the  examinees  with  regard  to  their  atti¬ 
tude  toward  co-workers  and  subordinates,  and  if  we  should  assume  that  the 
technical  skills  tasks  reflect  a  strong  task  orientation  while  confoat  or 
tactical  skills  tasks  reflect  more  concern  or  consideration  for  the 
motivations  and  feelings  of  men,  then  we  might  begin  to  Interpret  the 
flndlngseln  terms  of  leadership  models  such  as  the  one  proposed  by 
Fiedler -S'  Obviously  the  assumptions  are  shaky  at  best,  and  much  work 
would  need  to  be  done  to  demonstrate  any  such  comparability. 

Even  though  there  is  some  consistency  in  patterns  of  relationship, 
the  internal  analysis  conducted  earlier  and  reported  in  Research  Memo¬ 
randum  68-10  casts  serious  doubt  on  the  structure  of  this  test  in  terms 
of  its  measuring  any  clear  patterns  of  attitude  estimation  by  the  examinee. 
The  low  coefficients  of  correlation  reported  herein  cast  further  doubt  on 
the  operational  value  of  this  particular  approach  to  the  measurement  of 
leadership  potential.  The  results  of  these  two  analyses  suggest  that 
very  little  of  value  would  be. gained  in  making  present  GAT  measurement 
procedures  an  operational  part  of  officer  assessment. 


Fiedler,  F.  B.  Personality  and  situational  determinants  of  leadership 
effectiveness.  In  D.  Cartwright  and  A.  Zander  (Eds.),  Group  dynamics. 
(3rd  ed.)  New  Fork:  Harper  and  Row,  1968. 
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CORRELATION  BETWEEN  ORIGINAL  48  GAT  SCALE  SCORES 
AND  OEC  TASK-SPECIFIC  FACTOR  SCORES 
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Cynical  View  of  Htaan  Nature  29  -.14  -.12  .12  37 
Educated  Sophistication  30  .10  .15  .12  38 
Personal  Integrity  31  .15  .10  39 
Activities  32  .13  .10  40 
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ISR  -  Improper  Supply  Records,  AI  -  Automotive  Inspection,  and  AL  -  Airfield  Layout 


IS*  -  Improper  Supply  Records,  AI  -  Autoaofclve  I aspect Ion ,  and  AL  -  Airfield  Layout 


TABU  7 


PATTONS  OP  SIGNIFICANT  COMELATIONS  BETWEEN  GAT  S COKES  AND 
TOTAL  8C0KES  POK  THE  CGMflJNI  CATIONS  EXHIBIT  AND  MARCH  ORDER  TASKS 
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APPENDIX 


Table  A-l 


GAT  ITEMS  INCLUDED  IN  NEW  ITEM  SUM  SCALES 
CONSTRUCTED  SPECIFICALLY  FOR  THIS 
VALIDITY  ANALYSIS 


Scale  Name  Items  Included* 

Social  Desirability  1  3,  9 *,  10,  15,  29*,  25*,  30,  31,  54,  39* 

43,  45*,  54,  55,  58,  63,  66,  67*  68,  69,  75*, 
74,  75 


NCO  Unique  17,  24*,  41,  49,  50*,  61* 

Recruit  Unique  11*,  26,  27,  29*,  35,  42,  44*,  46,  52,  55*, 

56,  70 


ROTC  Unique  6*,  12*,  I7*,  18*,  19*,  21*,  23*,  24,  28*,  32*, 

35*,  36*,  37*,  40*,  51*,  56*  57,  59*,  60,  61, 
70* 


“Starred  items  (*)  were  scored  in  reversed  direction  so  that  highest 
value  would  be  given  to  the  positive  or  socially  desirable  response. 
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